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ABSTRACT 

This study attempted to identify potential measures 
for evaluating the multiple outcomes and activities associated with 
programs designed to promote employment of youths with disabilities. 
It sought to identify measures for 17 program outcomes and 51 
associated activities that had been identified previously by model 
demonstration transition project directors. Thirty project directors 
suggested both qualitative and quantitative measures for the 
evaluation of every outcome and activity. Also created was an 
in-depth task analysis of the 17 program outcomes and 51 associated 
activities and strategies associated with each outcome and activity. 
The study extended use of the systems-change model previously 
introduced by the authors. Tables and figures present study details 
including: employment-focused outcomes with the highest mean ratings; 
indicators of selected outcomes and activities across four conceptual 
levels (individual/family, program, organizational, at.** community); 
and conceptual and analytical models used. An appendix lists 
indicators for all outcomes and activities. (Contains 27 references.) 
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School to Work Transition: Identification of Employment-Related Outcome 

and Activity Indicators 1 

For the past decade, employment of youths with disabilities has been a primary focus of 
policy makers, researchers, and service providers. Each conceptual model of transition that 
emerged featured employment as a desirable outcome (e.g., Halpern, 1985, 1992; Wehman, 
Kregel, & Barcus, 1985; Will, 1984). During this period, researchers have assessed continuously 
whether or not youths have achieved employment after leaving school (e.g., de Bettencourt, 
Zigmond, & Thornton, 1989: Fardig, Algozzine, Schwartz, Hensel, & Westling, 1985; Haring, 
Lovett, & Smith, 1990; Hasazi, Gordon, & Roe, 1985; Mithaug, Horiuchi, & Fanning, 1985; 
Roessler, Brolin, & Johnson, 1990; Wagner, 1989) and policy makers have funded programs as 
demonstrations in promoting employment (Individuals with Disabilities Education Act 
Amendments of 1983; 1986; 1990). It is recognized that secondary-level school programs, in 
conjunction with other community organizations, must establish interventions that will lead to 
and result in employment of youths with disabilities. 

Recently, specific practices thought associated with positive employment outcomes have 
begun to emerge, although empirical support for such practices is not abundant (e.g., Kohler, in 
press; Rusch, DeStefano, Chadsey-Rusch, Phelps, & Szymanski, 1992). It has been much easier 
to evaluate the employment outcomes of students than it has been to identify programmatic 
outcomes associated with producing employment. Program evaluation has focused typically on 
outcomes achieved by program participants, but in many cases has overlooked specific 
documentation of the intervention, or levels of the intervention, that can be used to assess 
particular program elements in relation to participant outcomes. Thus, programs may be 
identified as effective or exemplary, yet the cause of effectiveness may be unclear (see Kohler, 
DeStefano, Wermuth, Grayson, & McGinty, in press). 

One problem associated with determining program effectiveness is the lack of agreed 
upon outcomes and well-defined activities implemented in conjunction with the desired 
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outcomes (Bruininks, Wolman, & Thurlow, 1990; DeStefano & Wagner, 1992; Halpern, 1990; 
Oakes, 1986; Rusch, Kohler, & Hughes, 1992). Further, there is a need for measures with which 
to assess the level and degree of intervention (activities) and achievement (outcomes). Another 
complicating factor involves the multiplicity of outcomes that might be achieved by programs 
focused on employment. Evaluation and social science literature suggest that programs operate 
in the context of multiple stakeholders and or systems that may complicate or make demands 
upon the program (Rappaport, 1977; Worthen & Sanders, 1987). Thus, outcomes related to the 
various stakeholders or systems should be considered when evaluating program effectiveness. 
Finally, these multiple outcomes would subsequently affect, or be related to, outcomes 
associated with students. 

The primary purpose of this study was to identify potential measures for evaluating the 
multiple outcomes and activities associated with programs designed to promote employment of 
youths with disabilities. Specifically, this study sought to extend the work of Rusch, 
Enchelmaier, and Kohler (in press) by identifying measures for 17 program outcomes and 51 
associated activities that had been identified previously by model demonstration transition 
project directors from across tne United States. 

Rusch et al. (in press) identified 22 employment-related outcomes and 65 associated 
activities believed to be important by model demonstration project directors across the United 
States. Through a two-round Delphi procedure, 106 and 75 project directors, respectively, rated 
the outcomes and activities for importance on a nine-point Likert-type scale. The outcomes and 
activities were organized according to the systems-level conceptual framework originally 
conceived by Rusch and Phelps (1987) and used in previous analyses of model demonstration 
final reports to identify project purposes, activities, outcomes, and barriers (e.g., Rusch, Kohler, 
& Hughes, 1992). This framework consists of four levels of possible influence, and thus 
suggests that programmatic outcomes occur and impact more than the individuals participating 
in a particular program. Rusch, Kohler, and Hughes (1992) also suggested that programs 
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focused on employment outcomes may have to achieve outcomes across all levels to produce 
meaningful, systemic change. The four levels include (a) the student and family, typically the 
primary focus of the program or intervention; (b) the pro gram responsible for administering the 
intervention; (c) the or ganization s that collaborate with the program to provide services; and (d) 
the community, which includes all the generic services, opportunities, and barriers typically 
taken for granted in defining the context of a program. The conceptual framework is 
graphically represented in Figure 1. 

Within each conceptual level, a number of outcomes were identified; for each outcome, a 
number of activities thought associated with promoting the outcome were identified also. For 
example, at theStudent/Fami!y level, Outcome 1 stated "Model transition-to-employment 
projects must place students into competitive, integrated employment (including supported 
employment)." Activities associated with this outcome included: (a) Provide job placement 
services, (b) work with adult service agencies to ensure placement, (c) provide job exploration 
and job-training opportunities as part of the school curriculum to prepare students for 
competitive employment, (d) provide job support services, and (e) provide the technical 
assistance to adult service agencies to provide job placement and job support services (Rusch et 
al., in press). Across the four conceptual levels, Rusch et al. (in press) reported that the mean 
ratings of the 22 outcomes ranged from 5.95 to 8.77. The current study focused on the 17 
outcomes that received a mean rating of 7.00 or higher (see Table 1). For those five outcomes 
rated less than 7.00, there was less agreement as to importance, as well as greater response 
variability. 



Insert Table 1 and Figure 1 about here 



This study sought to identify measures that would serve as indicators for each outcome 
and activity. Thus, the intent was to extend the analytic model that featured outcomes across 
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multiple levels, identify measures that would indicate the outcome had been attained or 
achieved, identify activities associated with producing related outcomes, and identify measures 
that would indicate the activity had been implemented, and in some cases, the level of 
implementation. Figure 2 illustrates a model of the perceived organizational relationships 
between the outcomes, activities, and their measures. 

Method 

P articipants 

Directors of OSERS-funded model demonstration projects focused on employment 
served as the participant pool for this study. A letter was mailed to the 167 project directors 
identified to participate in the Delphi procedure referred to previously. The letter requested 
that they participate in a study to identify measures for reporting the outcomes and activities. 
Fifty-three project directors returned a postcard indicating their willingness to participate. 
Subsequently, an instrument that listed each of the 17 outcomes was mailed to the 53 
responders. They were asked to identify the five outcomes for which they were most interested 
in identifying measures, and to rate these outcomes from 1 to 5, with 1 being their first priority. 
Forty-nine project directors responded to this request. 
Data Collection 

For each outcome, a list of participants wanting to focus upon the outcome was 
generated, by priority. Using these data, participants were assigned to work on specific 
outcomes and the associated activities. A minimum of three participants were assigned to each 
outcome; all participants were assigned to work on the outcome they selected as their first 
priority. In some cases, participants were assigned to focus on their subsequent priorities as 
well, since these outcomes were not selected as a first priority by three individuals. For 
example, 21 participants selected Outcome 1 as a first priority, 2 individuals identified 
Outcomes 3, 8, and 17 as a first priority, and no one selected Outcomes 4, 9, 11, or 12 as their 
first priority. Final assignment included the following distribution of participants to outcomes: 
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1 outcome with 21 participants, 4 outcomes with 4 participant each, and 12 outcomes with 3 
participants each. Finally, individual participants were assigned to focus on a range of 
outcomes: 29 participants were assigned 1 outcome, 15 were assigned 2 outcomes, and 5 were 
assigned 3 outcomes. 

The instrument utilized in the Rusch et al. (in press) Delphi procedure was modified for 
use in the present investigation. This instrument was divided into four sections, one for each 
conceptual level (e.g, Student/family, etc.). Within each section, the outcomes and their 
associated activities were listed. Space for writing in suggested measurements or indicators 
followed each outcome and activity. The authors generated initial suggestions for the first three 
outcomes to serve as examples to guide the participants. These examples were initially drawn 
from final reports and then were circulated to research faculty and staff of the Transition 
Research Institute at Illinois whose feedback was used to develop the final examples. 

Subsequently, a letter, instructions, the instrument, and a return envelope were mailed 
to the 49 participants; each participant was instructed to work on specific outcomes, but was 
encouraged to suggest measures for any of the others as well. Approximately six weeks later, a 
reminder letter was mailed to all participants who had not responded. As responses were 
received, a running list of suggested measures was compiled for each outcome and activity. 
Content analysis was conducted on the data; redundant measures were deleted and the list was 
clarified, A draft list of suggested measures was produced and mailed for feedback to the 167 
project directors identified initially as the participant pool. 

Results and Discussion 

Thirty participants (61.2%) returned suggested measures; suggestions were received for 
every outcome and every activity. Participants represented 11 OSERS' funding competitions, 
such as Handicapped Children's Model Programs: Youth Employment Projects (84.023D) and 
Secondary Education and Transitional Services: Training and Employment Models for Youth 
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with Severe Handicaps (84.158N). Also, participants represented 24 states across all regions of 
the United States. 

Both qualitative and quantitative indicators were submitted. For example, qualitative 
measures suggested to indicate that students had been trained with peers who do not have 
disabilities (Outcome 3) included the identification of roles played by peers and identification of 
the natural interactions and supports present at the setting. Quantitative measures for the same 
outcome included the hours and proportion of contact per day with nondisabled peers, and the 
number and proportion of such peers present in classroom settings. One outcome from each 
conceptual level, associated activities, and suggested measures are outlined in fable 2^. 

Insert Table 2 about here 



Two responses were received after the draft document was circulated to the 167 project 
directors for review. Both of these responses were very positive and focused on the 
comprehensive nature of the outcomes, activities, and measures and their usefulness in 
planning, proposing, and evaluating projects and services. 

This investigation identified measures for 17 outcomes and 51 related activities believed 
to be important in promoting employment of youths with disabilities. These findings extend 
previous research by identifying qualitative and quantitative indicators for use in identifying 
and reporting program outcomes and related activities. This investigation directly extends the 
Rusch et al. (in press) study, and the literature in general, by identifying hundreds of specific 
measures for the 17 outcomes and 51 activities that had been validated by model demonstration 
transition project directors as important factors of programs focused on employment. 

Most importantly, this investigation created an in-depth task analysis of the 17 program 
outcomes and the 51 activities thought to enhance achievement of these outcomes. In essence, 
what initially began as a list of measures to be used as indicators that an outcome had been 
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achieved or an activity had been implemented emerged as a comprehensive representation of 
strategies associated with each outcome and each activity. In other words, in looking for ways 
to measure that an activity had occurred, what emerged was a list of strategies associated with 
that activity. For example, Outcome 6 states that "Model transition projects should utilize 
individualized transition planning for students with disabilities/' Activity A for this outcome 
states: "Develop strong linkages with vocational rehabilitation services to develop the IEPs." 
Not only does the list of suggested measures provide ways to indicate that this activity 
occurred, it suggests strategies or actions that make up the activity (e.g., work with the 
vocational rehabilitation system). The implication is that in developing agency linkages, 
agreements need to be developed, consultations should occur, various personnel will 
participate in developing and signing the IEP, vocational rehabilitation counselors will contact 
and provide services to students, and information will be shared across agencies. Thus, for 
every outcome and activity, a number of strategies to implement the activity or enhance the 
outcome have been identified. 

Further, the indicators identified in this investigation should enhance research efforts to 
identify effective transition practices. In order to identify evidence that particular practices are 
associated with positive student outcomes, data across programs and contexts must be collected 
and analyzed in relation to these outcomes. The measures identified in this investigation 
provide an array of variables for use in future research. Also, since the outcomes, activities, and 
measures are somewhat specifically defined, data collection can occur in numerous sites and 
the results pooled and compared to evaluate relationships between program activities and 
outcomes and student outcomes. 

The analytical model applied in this study provides a tool for conceptualizing 
relationships between outcomes, activities, and indicators. The Individuals with Disabilities 
Education Act Amendments of 1990 forces us to focus on specific outcomes and to develop a 
"coordinated set of activities" (104 STAT. 1103). To achieve the full intent of the legislation- 
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post-school success for students-the field must identify those activities that result in positive 
outcomes. To do this, we must evaluate our interventions, in part and in whole, and measure 
outcomes. In Figure 3, the analytical model has been extended specifically to Outcome 1 
pertaining to competitive, integrated employment, using examples of measures identified in 
this investigation. Similar models for each outcome and activity using their related measures 
could be developed to visually represent a preliminary research or evaluation model. Finally, 
application could be extended to evaluate relationships among outcomes across the four 
conceptual levels. 

Importantly, this study also extended further the systems-change model introduced by 
Rusch and Phelps (1987) and later used to analyze the multiple outcomes of model 
demonstration transition programs. In previous applications (Rusch, Kohler, & Hughes, 1992; 
Rusch et al., in press; Rusch, Kohler, & Rubin, submitted for publication) it was suggested that 
program developers and those focused on restructuring educational programs consider 
outcomes beyond the student in order to achieve the greatest impact on student outcomes. 
Suggested measures for multiple outcomes and activities across the four conceptual levels were 
identified in this study, thus providing numerous specific examples for consideration. Further, 
in each of the previous studies, it was found that few community-level outcomes or activities 
had been achieved (Rusch, Kohler, & Hughes, 1992; Rusch et al., in press; Rusch et al, 
submitted for publication). In the current investigation, indicators (and subsequently, 
strategies) were identified for two outcomes and six activities at the community-level. It is 
interesting to note that 21 (70%) of the 30 participants in the current investigation selected 
Outcome 1 as their first selection on which to focus. No other outcome had more than four 
participants identify it as a first priority. Specifically, for 10 of the 14 Program, Organizational, 
and Community-level outcomes, two or less persons selected them to focus on, thus illustrating 
the challenging and complex nature of the task. As indicated in the research literature, the 
primary focus in the past has been on evaluating student outcomes (i.e., employment or 
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residential status, postsecondary education participation, social participation). It is much more 
difficult to measure program, organizational, or community outcomes. 

The findings of this investigation are subject to some limitation. Although a number of 
indicators were identified for each outcome and activity, the list is by no means exhaustive. 
Researchers and service providers should not feel limited by these findings, but rather should 
use them as a starting point for implementation of services and evaluation of relationships 
between outcomes, and outcomes and activities. Also, since only 30 participants were involved 
in the identification of potential measures, the list is somewhat limited to their contexts and 
experiences. Future research is needed to expand the number of possible stakeholders who 
identify indicators. However, since these 30 participants represented model demonstration 
programs from across the United States and across a number of funding competitions, the 
findings may generalize to non federally-funded programs focused on employment. Further, it 
should be recognized that many of the measures identified in this investigation are not "new," 
and we do not mean to suggest that they are. What is important is that 30 model demonstration 
project directors worked together to compile both current and innovative ways to measure 
whether or not 17 employment-related outcomes and 51 activities have been achieved. Finally, 
we must realize that although the analytical model applied in this study to visually represent 
the relationships between outcomes, activities, and their related measures is somewhat 
simplified, the relationships are quite complex. 

It has become more and more apparent that numerous forces or variables relate to the 
achievement of positive employment outcomes for youths with disabilities. The present 
investigation offers a complex array of indicators and strategies for implementing and 
evaluating program outcomes and activities across four conceptual levels. As we restructure 
our secondary-level special education programs and services, we must think beyond what is 
occurring in an individual classroom or employment site. The coordinated sets of activities that 
we develop to prepare students for post-school life must be supported by evidence of 
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effectiveness and must impact programs, organizations, and the community, as well as 
students. 
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This research was sponsored in part by the Office of Special Education and Rehabilitative 
Services (OSERS), U. S. Department of Education, under a cooperative agreement (H158- 
T-00-1) with the University of Illinois. Opinions expressed herein do not necessarily 
reflect those of OSERS. 

A complete list of outcomes, activities, and suggested measures is included in Appendix. 
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Table 1 



Employment-focused Outcomes with Mean Ratings of 7.00 or Hig her 



Conceptual Level 


Number 


Outcomes 


Student/Family 


1 


Model transition-to-employment projects must place students into 

ComT5PtitivP intppratpd pmnlnvmpn^ ( infltiHino - Qiinr>ortpH 

wiillA.UU VC/ 1 1 1 H» EL 1 C* lt\4 LUIUIUV UH_lh. \ 1 1 UU 11 IfL jUULA/1 vn-vI 

employment). 




2 


Model transition-to-employment projects should demonstrate 
functional skill development of students. 




3 


Model transition-to-omnlovmont nrniort^ should onmiro that shidpnts 

are educated with their non-disabled peers. 


Program 


4 


\^oHp1 tranQitlOT*i-to-Pmnlr\\/mr>r>t nrniPPt*; cVinnlH rnihlich ror\rvrfrc r\( 

1V1UUC1 IICIIOIIIUII KKJ 1 1 1 IJ1UV 1 1 Id 1 1 IJlUlv^lO DIIUL1114 IJliCllOll ICUU' lo \JI 

their students' documented progress. 




5 


A^oHpI trarisitiori-to-prnr'ilov/rnpnt nrniPPtQ chnnlH Vvo /"nrttirtnpH Vvn/riiiH 

lYlUUtl 11 CXI lOl lllsl 1 WJ 1, 1 1 ILHUV 1 1 U.1 1 1 VJl\Jl\ r \.KJ DllLrLllV*! LA, (111 (Lid! LAjyUIivJ 

the federal funding period. 


• 


6 


Model transition projects must i tilize individualized education 
planning in relation to transition. 




7 


Model transition-to-employment projects should provide students with 
job skill training. 




8 


Model transition-to-employment projects should document student 
progress in employment-related skills (e.g., social skills). 




9 


Model transition-to-prnnlnvrnpnt nroiprts should .*»rhipvp rpnliration at 

least at the level of full utilization of a project feature (e.g., interagency 
teaming), component (e.g., placement), or product (e.g., a training 
manual). 




10 


Model transition-to-employment projects should establish employment 
support services. 




11 


Model transition-to-employment projects should undertake 
development of materials to facilitate teplication (e.g., replication 
guides, training manuals, assessment instruments). 




12 


Model transition-to-employment projects should demonstrate cost 
effectiveness. 
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Table 1 (continued) 



Conceptual Level Number 



Outcomes 



Organization 13 Model transition projects should disseminate information about their 

projects by producing a product at least at the level of an article foi the 
popular press. 

14 Transition-to-employment projects should develop and document a 
formal interface between education and community services (e.g., 
between schools and state vocational rehabilitation agencies). 

15 Model transition-to-employment projects should develop and 
document a cooperative service delivery model where more than one 
agency is providing consumer services. 



Community 



16 Model transition-to-employment projects should demonstrate 
improved access to community-based services for students. 

17 Model transition-to-employment projects should demonstrate 
improved work opportunities for students. 



Note . The data in this table are from "Employment outcomes and activities for youths in 
transition 1 ' by F. R. Rusch, J. F. Enchelmaier, and P. D. Kohler, in press, Career Development for 
Exceptional Individuals . Copyright by the Division on Career Development and Transition of 
the Council for Exceptional Children. Reprinted by permission. 
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Table 2 

Indicators of Selected Outcomes and Activities Across Four Conceptual Levels 



NUMBER 



DESCRIPTION 



INDIVIDUAL/FAMILY LEVEL 

Outcome 1 Model transition-to-employment projects must place students into competitive, 

integrated employment (including supported employment). 

Indicators 

• Hours worked 

• Hourly wages 

• Number of students placed 

• Student demographics 

• Job types 

• Length of employment (days, weeks, months) 

• List of benefits (i.e., medical, vacation, sick time, profit sharing) 

• Number of jobs held prior to graduation and summary of evaluations of each 
placement 

• Annual salary 

• Number of full-time and part-time positions 

• Job patterns (e.g., never changed, voluntary change, laid off, quit, etc.) 

• Student satisfaction measures 

• Termination, reason (elaborate) 

• Record of how job was initially obtained and by whom 

• Number of employees at each business 

• Number and ratio of employees without disabilities on site when student present 

• Record of job match to student-stated work interests 

• Record of job advancement following initial placement — increased job 
responsibilities or increased wages (e.g., in hrs worked, raises, promotions, job task 
responsibilities; increased level of indispensability to employer) 

• Documentation of single-subject research study 

• Reduction in public assistance (e.g., SSI, workers comp, public aid, etc.) 



Activity la 



Provide job placement services. 



ERLC 



Indicators 

• Potential Employer Contact Log (# of entries) 

• Number of potential employers 

• Number of student interviews 

• Number of placements 

• Types of placements 

• Number of potential employees 

• Identification of natural supports in each placement 

• Documentation of a specific "marketing" plan for each student 

• Student/family job preferences 

• Number employers contacted and method 

• Number student interviews, alone or assisted 

• Number of successful and unsuccessful placements and whether placement was in 
the top-priority list of student or family 

• Trainee waiting lists for placement 

• Number of placements per student 
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Activity lb Work with adult service agencies to ensure job placement. 

Indicators 

• Signature of adult service representatives on each IEP 

• Record or letter of interagency agreement 

• Referral documentation 

• Number of meetings and frequency 

• Record of joint projects (i.e., Job Fair) 

• Record of business involvement with adult service agencies 

• Identification of roles adult service agencies play in job placement 

• Log of case management activities by type and frequency 

• Record of transition plan updates with adult service agencies 

• Written statement of percent time commitment to student and family that will 
occur at age 21 

• Number of meetings of adult agencies with school personnel and/or student/ 
family 

• Written progress reports on placement efforts 

• List of transition team members 

Activity lc Provide job exploration and job training opportunities as part of the school 

curriculum to prepare students for competitive employment. 

Indicators 

• Number of days training in community (unpaid) per week 

• Number of days paid work per week 

• Hou^s worked during school day 

• Record of training and employment sites 

• Record of peer job coaching 

• Record of gifted and talented student support 

• Record of all school personnel involved in curriculum 

• Number of hours 

• List of job exploration sites, general and specific purposes for each site, 
competencies gained per student during exploration, number of hours in job 
exploration, and type of site 

• Performance data per student 

• Data on level of supervision 

• List of products produced and quantity (e.g., student resumes, training plans, or 
work profiles) 

• Number of training sites per student 

• Documentation of individualized training programs for students that reflect 
systematic instruction and strategies 

• Baseline and probe data pertaining to training 

• Graphs of student performance 

• Documentation of types of instruction or training provided 
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Activity Id Provide job support services. 

Indicators 

• Number of training hours provided by teacher 

• Number of hours teacher present at employment site 

• Record of off-site support provided by teacher 

• Number of observatr hours provided by teacher 

• Record of support piovided by others (i.e., job coaches, voc. rehab, counselors, job 
developers) 

Number of hours by school personnel other than teachers 
- Record of support provided by parents 

• Record of support provided by employer or supervisor 

• Contacts with family 

• Hours and type of school-based job-related instruction 

• Record of data-based fading against targeted performance criteria 

• Number of employee evaluations done by employer per month 

• Record of support provided by co-workers 

• Record of unusual incidents and training interventions 

• Record of training strategies and procedures 

• Record of data collection documenting skill acquisition 

Activity le Provide technical assistance to adult service agencies to provide job placement and 

job support services. 

Indicators 

• In-service workshop hours directed toward teaching job-placement and support 
techniques 

• Hours of direct training provided to job coaches on the job 

• One-to-one contact (contact logs) 

• Evidence of curriculum used 

• Type of training and technical assistance activities provided 

• Record of funds spent to train adult service providers 

• Workshop or inservice evaluation data 

• Needs assessment data 

• Number of people trained 

• List of training materials developed 
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PROGRAM LEVEL 

Outcome 6 Model transition projects should utilize individualized transition planning for 

students with disabilities* 

Indicators 

• Number of plans developed 

• Number of plans completed 

• Written documentation of transition sendees as part of IEP 

• Record of number of transition planning meetings 

• Signatures of adult service providers and community agency personnel on IEP 

• Compilation of types of transition outcomes for students on annual basis 

• Results of parent or family survey of planning process 

• Evidence from student files illustrating program modifications to meet student's 
individualized transition needs 

• List of agency representatives participating in planning 

• Number of transition objectives in IEP 

• Documentation of assessment information utilized in plan development 

• Documentation of parent involvement in planning 

• Documentation of student involvement in planning 

• Documentation that service or curricular activities were provided as stated in the 
student's plan 

• Evidence of curriculum that facilitates student involvement in planning t 

Activity 6a Develop strong cooperative linkages with vocational rehabilitation services to 

develop the IEPs, 

Indicators 

• Number of agreements developed 

• Number of consultations 

• Documented participation of personnel 

• Signature of vocational rehabilitation personnel on IEP 

• Record of contacts between vocational rehabilitation and student 

• Report of number of students receiving services from vocational rehabilitation 

• Letters of agreement 

• Documentation of joint use of information (e.g., school records, assessment 
information, medical data) 

• Evidence of referral system or process 

• Documentation of services provided to students 
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Activity 6b Develop strong cooperative linkages with vocational education services to develop 

the lEPs. 

Indicators 

• Record of agreement(s) developed 

• Number of joint activities, meetings, or consultations 

• Record of actual participation in IEP development 

• Signature of vocational education personnel on IEPs 

• Report of number of goals or objectives contained in IEP carried out by or in 
conjunction with vocational education 

• Record of vocational education contact with families 

• Vocational education services identified on IEP 

• Record of attendance at meetings 

• Record of interagency agreements 

• Record of student enrollment in vocational curricula 

• Record of vocational education services provided 

• Evidence of collaborative consultation between voc ed and special ed 
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ORGANIZATION LEVEL 

Outcome 15 Transition-to-employment projects should develop and document a cooperative 

service delivery model when more than one agency is providing consumer services. 

Indicators 

• Number of agencies providing services 

• Organizational structure for service provision 

• Interagency agreement(s) 

• Record of referral among agencies 



Activity 15a Articulate the roles of all associated agencies. 
Indicators 

• Evidence of a process for developing of collaborative agreements 

• Documentation of collaborative agreements 

• Documentation of contractual arrangements 

• Evidence of a process reviewing roles on a regular basis 

• Number of formal operating agreements 

• Number of informal operating agreements 

• Evidence of networking effectiveness 

• Evidence of a designated "coordinating" agency to oversee local agencies 

• Number or existence of state laws or regulations reducing barriers to agency 
collaboration or articulating collaborative roles 

• Documentation of services provided by and efforts of each agency 

Activity 15b Employ personnel whose role is to coordinate project activities. 
Indicators 

• Job description of project coordinator or manager 

• Evidence of a "coordinating" agency to oversee local agencies 

• Documentation of state and local funds earmarked to support coordinating agency 
in this task 

• Evidence of agency and project funding of personnel 



Activity 15c Document services provided by cooperating agencies. 

Indicators 

• Record of employment services provided by type and by student 

• Record of community living facilities and/or sendees provided 

• Record of transportation arrangements and services 

• Assignment of coordinating personnel, agency, or local planning councils to collect 
specific data on services offered, clients, costs, etc. 

• Analysis of data collected for future decision making 

• Case history of clients 

• Number and type of activities completed by caseworkers 

• Evidence of a process for evaluating accountability of cooperating agencies 
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COMMUNITY LEVEL 

Outcome 17 Model transition-to-empioyment projects should demonstrate improved work 
opportunities for youths with disabilities. 

Indicators 

• Record of types of jobs that comprise placements 

• Statistics pertaining to wages, benefits, and hours 

• Percent of graduating class employed by level of employment (i.e., full-time, part- 
time) 

• Percent employed at or above minimum wage 

• Percent who move to improved work situations (e.g., promotions, job changes for 
increased salary, benefits, working hours, etc.) 

• Percent who lose jobs and/or move to "poorer" jobs 

• Number of employers associated with project or program 

• Number of students employed first year of project compared to subsequent years 

Activity 17a Evaluate and document effectiveness of job placement and maintenance activities. 

Indicators 

• Length of time on job 

• Employee satisfaction data concerning job placement, pre-/post- project 

• Employer satisfaction data concerning job placement, pre-/post- project 

• Documented opportunities for advancement 

• 1-, 3-, 5-year follow-ups on youths: 

1. Employed in jobs for which training was provided; in jobs for which training 
was not provided 

2. Employed but changed job (up and down) in job trained; not in jobs trained 

3. Unemployed; never employed; previously employed 

• Data on youths employed and wages, benefits, length of employment, pre-/post- 
project 

• Data on family satisfaction with job placement, pre-/post- project 

• Data on employer willingness to hire, pre- /post- project 

Activity 17b Research job trends and business requirements. 

Indicators 

• Record of project cr employer advisory committee, members, meetings 

• Labor-market surveys: 

Stable employment opportunities 

Potential increased employment opportunities 

Decreasing employment opportunities 

Dead-end employment 

Career ladder employment 

Job requirements 
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Activity 17c Work cooperatively with community agencies to conduct longitudinal studies. 

Indicators 

• Assess student outcomes: employment, community living, recreation, etc. 

• Assess quality of life via self rating or reliable informant 

• Follow-up data on individuals who change agencies 

• Evaluation data pertaining to working relationship between agencies and project 

• Record of agreements detailing longitudinal studies to be conducted and roles of 
participants 

• Documentation of funding source 

• Evidence of research design for study 



* From: Kohler, P.D., & Rusch, F.R. (1993). School to work transition: Identification of employment- 
related outcome and activity indicators. Champaign: University of Illinois, Transition Research 
Institute. (Submitted for publication.) 
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Organization 



Community 




Fi gure 1 . Systems-level conceptual framework for evaluating program activities and outcomes. 
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Outcome 



General statement of results; 
the product of action 




Activity 

Specific statement; 
action associated with 
producing outcome(s) 



Indicator(s) that activity 
has taken place 




Activity 

Specific statement; 
action associated with 
producing outcome(s) 




Indicator(s) that activity 
has taken place 




Activity 

Specific statement; 
action associated with 
producing outcome(s) 



— (Indie 



Indicator(s) that activity 
has taken place 



Fi gure 2 . Analytical model illustrating perceived organizational relationship between an 
outcome, activities, and indicators. 
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Outcome 

Competitive, integrated 
employment 



( Hours worked 1— 
^^^^^ouriy wages^^^— 




^^^escription of benefit^^ 



Activity 



Provide job placement 
services 




Activity 



Work with adult service 
agencies to insure job 
placement 




Activity 

Provide job support 
services 




Fi gure 3 . Analytical model of outcome, activities, and indicators extended to integrated, 
competitive employment. 
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NUMBER DESCRIPTION 



INDIVIDUAL/FAMILY LEVEL 

Outcome 1 Model transition-to-employment projects must place students into competitive, 

integrated employment (including supported employment). 

Indicators 

• Hours worked 

• Hourly wages 

• Number of students placed 

• Student demographics 

• Job types 

• Length of employment (days, weeks, months) 

• List of benefits (i.e., medical, vacation, sick time, profit sharmg) 

• Number of jobs held prior to graduation and summary of evaluations of each 
placement 

• Annual salary 

• Number of full time and part time positions 

• Job patterns (e.g., never changed, voluntary change, laid off, quit, etc.) 

• Student satisfaction measures 

• Termination, reason (elaborate) 

• Record of how job was initially obtained and by whom 

• Number of employees at each business 

• Number and ratio of employees without disabilities on site when student present 

• Record of job match to student-stated work interests 

• Record of job advancement following initial placement - increased job 
responsibilities or increased wages (e.g., in hrs. worked, raises, promotions, job 
task responsibilities; increased level of indispensability to employer) 

• Documentation of single-subject research study 

• Reduction in public assistance (e.g., SSI, workers comp, public aid, etc.) 

Activity la Provide job placement services. 

Indicators 

• Potential Employer Contact Log (# of entries) 

• Number of potential employers 

• Number of student interviews 

• Number of placements 

• Types of placements 

• Number cf potential employees 

• Identification of natural supports in each placement 

• Documentation of a specific "marketing" plan for each student 

• Student/family job preferences 

• Number employers contacted and method 

• Number student interviews, alone or assisted 

• Number of successful and unsuccessful placements and whether placement was in 
the top priority list of student or family 

• Trainee waiting lists for placement 

• Number of placements per student 
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Activity lb Work with adult service agencies to insure job placement. 

Indicators 

• Signature of adult service representatives on each IEP 

• Record or letter of interagency agreement 

• Referral documentation 

• Number of meetings and frequency 

• Record of joint projects (i.e., Job Fair) 

• Record of business involvement with adult service agencies 

• Identification of roles adult service agencies play in job placement 

• Log of case management activities by type and frequency 

• Record of transition plan updates with adult service agencies 

• Percent time commitment to student and family in writing that will occur at age 21 

• Number of meetings of adult agencies with school personnel, and/or student/ 
family 

• Written progress reports on placement efforts 

• List of transition team members 

Activity 1c Provide job exploration and job training opportunities as part of the school 

curriculum to prepare students for competitive employment. 

Indicators 

• Number of days training in community (unpaid) per week 

• Number of days paid work per week 

• Hours worked during school day 

• Record of training and employment sites 

• Record of peer job coaching 

• Record of gifted and talented student support 

• Record of all school personnel involved in curriculum 

• Number of hours 

• List of job exploration sites, general and specific purposes for each site, 
competencies gained per student during exploration, number of hours in job 
exploration and type of site 

• Performance data per student 

• Data on level of supervision 

• List of products produced and quantity (e.g., student resumes, training plans, or 
work profiles) 

• Number of training sites per student 

• Documentation of individualized training programs for students which reflect 
systematic instruction and strategies 

• Baseline and probe data pertaining to training 

• Graphs of student performance 

• Documentation of types of instruction or training provided 
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Activity Id 



Activity le 



Provide job support services. 
Indicators 

• Number of training hours provided by teacher 

• Number of hours teacher at employment site 

• Record of off-site support provided by teacher 

• Number of observation hours provided by teacher 

• Record of support provided by others (i.e., job coaches, voc. rehab, counselors job 
developers) J 

• Number of hours by school personnel other than teachers 

• Record of support provided by parents 

• Record of support provided by employer or supervisor 

• Contacts with family 

• Hours and type of school-based job related instruction 

• Record of data-based fading against targeted performance criteria 

• Number of employee evaluations done by employer per month 

• Record of support provided by coworkers 

• Record of unusual incidents and training interventions 

• Record of training strategies and procedures 

• Record of data collection documenting skill acquisition 

Provide technical assistance to adult service agencies to provide job placement and 
job support services. 

Indicators 

• In-service workshop hours directed toward teaching job placement and support 
techniques 

• Hours of direct training provided to job coaches on the job 

• One-to-one contact (contact logs) 

• Evidence of curriculum used 

• Type of training and technical assistance activities provided 

• Record of funds spent" to train adult service providers 

• Workshop or inservice evaluation data 

• Needs assessment data 

• Number of people trained 

• List of training materials developed 
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INDIVIDUAL/FAMILY LEVEL 

Outcome 2 Model transition-to-employment projects should demonstrate functional skill 

development of students. 

Indicators 

• List of social skills taught and acquired 

• List of work skills taught and acquired 

• Evaluation data from curriculum-based assessment instruments 

• Social validation data (e.g., targeted skills, performance levels, measures) 

• Evaluation data from performance in community-based activities 

• Skills assessment validated against employer requirements and work responsibilities 

• Observation data on skills taught, acquired, and generalized to community setting 

• Observation data on work skills taught, acquired, and generalized to community 
setting 

Activity 2a Use instruments and procedures that identify individual, functional skills, and 

consumer preferences and life goals. 

Indicators 

• Number and kind of IEP goals listed and achieved 

• Student signature on IEP 

• Data from situational assessment instruments (e.g., situations, skills, performance) 

• Employer assessments of student skills or performance 

• List of assessment instruments and/or procedures used 

• Parent survey data 

• Student survey data 

• Use of rehabilitation plan 

• Proportion and kind of IEP goals lifted and achieved 

Activity 2b Develop individualized objectives for students that reflect functional skill 

development in the domains of vocational skills, independent living, and community 
integration. 

Indicators 

• Number of objectives pertaining to vocational skills; number achieved 

• Number of objectives pertaining to independent living; number achieved 

• Number of objectives pertaining to community integration; number achieved 

• Documented match between IEP/ITP goals and objectives and stated transition 
outcomes 

• Number of students in job training, employment, or other situations which match 
IEP goals or activities 
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INDIVIDUAL/FAMILY LEVEL 

Outcome 3 Model transition-to-employment projects should ensure that students experience 

education or training with nondisabled peers. 

Indicators 

• Number and proportion of nondisabled individuals working in similar employment 
settings 

• Hours and proportion of contact per day with nondisabled peers 

• Number and proportion of nondisabled peers present in classroom settings 
(number of students with disabilities/number of students in classroom) 

• Record of extra-curricular activities with nondisabled peers (e.g., school or 
community clubs or activities) 

• Ratio of persons with and without disabilities at work site at time of work 

• Record of samples of self-initiated or other student initiated affiliations (e.g., 
evening telephone calls, joint activities) 

• Record of roles played by nondisabled peers 

• Record of natural interactions and supports present at the setting 



Activity 3a Conduct training activities for youths without disabilities as well as for those with 

disabilities. 

Indicators 

• Number and proportion of students without disabilities present during instruction 
provided to youths with disabilities 

• Number and type of training situations provided which include students with and 
without disabilities 

• Record of chronological age of peers involved in training activities 



Activity 3b Utilize integrated competitive and supported employment placements. 

Indicators 

• Number and proportion of students placed in individual integrated work settings 

• Ratio of people with and without disabilities at worksite during the time of work 

• Samples of coworker contact (e.g., breaks, work times, before and after work) 

• Number of coworkers providing support 

• Documentation of coworker support (e.g., functions, frequency) 

Activity 3c Utilize nonpaid volunteer placements in compliance with DOL standards. 

Indicators 

• Written letters of agreement 

• DOL worker permit (on site) 

• Number of students in volunteer placements 

• Number of hours worked 

• Number of volunteer sites 

• Number of letters or contracts of agreement 

• School permit for the group 

• Record of student duties or functions 

• Documentation of student outcomes as a result of placement 

• Record of student placement as an IEP or transition goal 
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Activity 3d 



Utilize community-based education and training sites. 



Indicators 

• Number of students provided with community-based instruction 

• Number of community-based instruction sites 

• Short description of sites and transportation used to access 

• Time spent at the sites for student; for age group; for disability group 

• Percent of school day in community-based instruction 

• Average number and proportion of hours in community-based instruction — across 
categories: work, residential (street, stores, etc.), leisure/recreation settings 

• Record of student achievement or performance in community-based instruction 

• Record of community-based instruction curricula 

• Record of community-based instruction objectives, criteria, and outcomes 
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PROGRAM LEVEL 



Outcome 4 



Activity 4a 



Activity 4b 



Activity 4c 



Model transition-to*empioyment projects should publish reports of their students' 
documented progress. 

Indicators 

• A list of the published reports 

• A mailing or recipient list 

• Timelines (evaluation plan) for prc 4 iration and dissemination 

Develop a final report for students and their families. 
Indicators 

• Sample comments pertaining to reports from students or families 

• Record of comments from a student/parent review committee 

• A list of the reports 

• A mailing or recipient list 

• Record of parent and student attendance at conferences during the program 

Utilize Individualized Education Plans (IEPs) as the basis for publishing reports. 
Indicators 

• Evidence of an IEP format that clearly documents progress 

• Evidence of a report format that clearly identifies areas on the IEP 

Conduct a longitudinal study of graduates and report these data. 
Indicators 

• Report of study design and sample selection 

• List of students in the sample 

• Reporting format for longitudinal data 

• Analysis of preliminary data 

• Student characteristics (e.g., months in SPED program, % time in resource 
programs, hrs. in vocational programs, demographics) 

• Employment status and environment 

• Employment outcomes (# weeks employed, hrs./wk, wages/hr., annual salary) 

• Current living status 

• Level of family involvement or support 

• Job types 

• Job history 

• Place of residence and cost 

• Satisfaction with social relationships 

• Maintain research data on graduates of school whether or not they participated in 
the program 
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PROGRAM LEVEL 

Outcome 5 Model transition-tn~empioyment projects should be continued beyond the federal 

funding period. 

Indicators 

• Evidence that project has met or exceeded majority (95%) of original goals and 
objectives 

• Evidence of commitment from local community to support (match percent 
funds/inkind) the revised or new goals of the project 

• Evidence of positive impact to local communities throughout the grant period 

• Record of staff employed and job descriptions 

• Letters of agreement with agencies 

• List of funding sources contracted to continue funding of program 

• Documentation of replication at other sites 

• Documentation of continuation plan 

• Documentation that program services will be provided by or transferred to another 
entity 

Activity 5a Identify alternative funding from other agencies to continue the project. 

Indicators 

• Identification and list of specific agencies who will contribute a percentage of 
funding (e.g., consortium of agencies to fund a transition center) 

• Identification of specific agency funding earmarked for transition-to-employment 
centers or services 

• Record of funds converted from other uses 

• Directory of potential funding sources 

Activity 5b Develop funding from within local special education program budget for transition- 

to-employment projects. 

Indicators 

• Record of personnel or matching funds identified and earmarked for project or 
services 

• Report of staff employed and job descriptions for which there is state or local 
reimbursement 

• List of and description of services or personnel billed to special education for 
transition-to-employment activities 

• Letters of agreement and proposed use of dollars 

• Record of agency linkages with local special education programs 

• Record of personnel and functions provided through special education budgets 
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PROGRAM LEVEL 

Outcome 6 Model transition projects should utilize individualized transition planning for 

students with disabilities. 

Indicators 

• Number of plans developed 

• Number of plans completed 

• Written documentation of transition services as part of IEP 

• Record of number of transition planning meetings 

• Signatures of adult service providers and community agency personnel on IEP 

• Compilation of types of transition outcomes for students on annual basis 

• Results of parent or family survey of planning process 

• Evidence from student files illustrating program modifications to meet student's 
individualized transition needs 

• List of agency representatives participating in planning 

• Number of transition objectives in IEP 

• Documentation of assessment information utilized in plan development 

• Documentation of parent involvement in planning 

• Documentation of student involvement in planning 

• Documentation that service or auricular activities were provided as stated in the 
student's plan 

• Evidence of curriculum which facilitates student involvement in planning 



Activity 6a Develop strong cooperative linkages with vocational rehabilitation services to 

develop the IEPs. 

Indicators 

• Number of agreements developed 
Number of consultations 
Documented participation of personnel 
Signature of vocational rehabilitation personnel on IEP 
Record of contacts between vocational rehabilitation and student 
Report of number of students receiving services from vocational rehabilitation 
Letters of agreement 

Documentation of joint use of information (e.g., school records, assessment 
information, medical data) 
Evidence of referral system or process 
Documentation of services provided to students 
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Activity 6b Develop strong cooperative linkages with vocational education services to develop 

the IEPs. 

Indicators 

• Record of agreement(s) developed 

• Number of joint activities, meetings, or consultations 

• Record of actual participation in IEP development 

• Signature of vocational education personnel on IEPs 

• Report of number of goals or objectives contained in IEP carried out by or in 
conjunction with vocational education 

• Record of vocational education contact with families 

• Vocational education services identified on IEP 

• Record of attendance at meetings 

• Record of interagency agreements 

• Record of student enrollment in vocational curricula 

• Record of vocational education services provided 

• Evidence of collaborative consultation between voc ed and special ed 
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PROGRAM LEVEL 

Outcome 7 Model transition-to-employment projects should provide job skill training. 

Indicators 

• Record of specific job skill competencies attained 

• Competency ratings of general work skills, job specific skills, and work-related 
behaviors 

• Placement rates in jobs related to training areas 

• Record of teaching objectives and student performance 

• List of job skill areas covered in training 

• Number of training hours 

• Number of follow-along hours 

• Evidence of business sector input in training program development 

• Evidence of local industry needs assessment relevant to employee competencies 



Activity 7a Conduct job skill analysis. 

Indicators 

• Data from direct observation of successful employees 

• Data pertaining to specific job tasks and general work skills 

• Number and type of jobs analyzed 

• Directory of job skill analysis 

• Evidence of job analysis based on best-practice criteria 

• Social validation data pertaining to production levels 



Activity 7b Develop a curriculum to facilitate training. 

Indicators 

• Evidence that curriculum goals and objectives relate directly to placement outcome 
measures 

• Data from continuous measures of student progress integrated throughout the 
curriculum 

* • Curriculum effectiveness based on placement outcome measures 

• Curriculum used 

• Revisions of curriculum used 

• External evaluation data pertaining to the curriculum 

• Content analysis based on best practice criteria 

• Number of auricular options available to students 

• Identification of curricular objectives and student competencies 

• Documentation of curricular development activities or process (e.g., funding, 
number of personnel involved, description of process) 



ERLC 



41 



Appendix 



131 



NUMBER 



DESCRIPTION 



Activity 7c 



Determine labor market needs. 



Indicators 

• Continuous sampling of labor market via employer interviews 

• Documented use of Department of Labor and state employment security data 

• Record of area employers on advisory committees 

• Lists of types of jobs in newspaper 

• Reports from job service 

• Data from labor market analyses 

• Data from chambers of commerce or Private Industry Councils 



Indicators 

• Data from employer interviews identifying and validating job skills 

• Job analysis data 

• Data from direct observation of employees at multiple sites 

• Record of job description analysis 

• Data from employer surveys 



Activity 7d 



Identify job skills that employers require of their employees. 




Appendix 
132 



NUMBER 



DESCRIPTION 



PROGRAM LEVEL 



Outcome 8 Model transition-to-employment projects should document student progress in 

employment-related skills (e.g., social skills). 

Indicators 

• Evidence of curriculum 

• Documentation of performance through self-report and employer evaluations 

• Documentation of assessment process or system 

• Documentation of process to apply assessment information to student program 

• Type of skills taught; how assessed; progress on objectives written for teaching the 
skill 

• Use of rehabilitation plan that tracks progress in target area over time 

• Number of IEP goals met or completed 

• Scores of standardized measures 

• Data from employer rating or evaluations 

• Data from self- rating by employee 

• Data from curriculum-based assessment 

• Data from situational assessment 



Activity 8a 



Undertake social skills assessment of students. 



Indicators 

• Record of instruments used or designed 

• Results per student 

• Record of role of teacher, student, employer and families in the assessment 
process 

• Documentation of barriers discovered in conducting assessments 

• Time and resources used in assessments 

• Evidence of rehabilitation plan based on staff or employer observations 

• Data from employer evaluations 

• Data from coworker or peer interviews 



Activity 8b Provide on-site community-based training in employment-related social skills. 

Indicators 

• Evidence of business involvement in training 
Record of coworker interactions 
Record of supervisor interactions 
Length of stay in job 

Number of students participating in training 
Record of skills taught and methods used 
Record of time and resources used for instruction 
Evidence of task analyses 

Percent of on-site time dedicated to social skills training 
Number and description of community sites used 
Record of persons providing on-site training 
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Activity 8c Assess student progress in job-related skills. 

In dicators 

• Amount of supervision required 

• Data on dependability and punctuality 

• Employer evaluation data 

• Documentation of progress in relation to IEP goals and objectives 

• Number of mastered objectives written to teach the skill or behavior 

• Employee or supervisor ratings pre and post instruction 

• Student self-report pre and post instruction 

• Record of barriers encountered in measuring outcomes 

• Record of IEP goals met or completed 

• Data from curriculum-based assessment 

• Data from situational assessment 

• Documentation of assessment process or system 

• Documentation of process to utilize assessment information 

• Standardized measures of adaptive behavior 

• Percentage of job tasks completed 

• Percentage of work completed compared to work required 

• Documentation of job supports needed to perform as required 
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PROGRAM LEVEL 

Outcome 9 Mode! transition-to-employment projects should achieve replication at least at the 

level of full utilization of a project feature* component, or product (such as a 
training manual). 

Indicators 

• Number of replication sites, components, or products 

• Letters of agreement 

• External evaluation data pertaining to replication sites 

• Evidence of model implementation checklist outlining principle model components 

• Number of sites in which on-site training was provided 

• Number of persons impacted by the replication 

• Record of impact pertaining to the replication 



Activity 9a Negotiate directly with an organization or agency to replicate the model project. 

Indicators 

• Number of contacts and record of meetings 

• Letter of agreement 



Activity 9b Develop and disseminate replication manual(s). 

Indicators 

• Copy of replication manual 

• Evidence from external review of th r manual; evaluation data 

• Number of manuals disseminated 

• Record of dissemination: who, when 

• Number of replicated projects utilizing manual 



Activity 9c Disseminate information and products to other agencies. 

Indicators 

• Number and type of information or products disseminated 

• Number of published articles 

• Number of responses to requests for information 

• Record of responses to requests for information 

• Evidence of dissemination process or system 

• Documentation of requests for information 
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PROGRAM LEVEL 

Outcome 10 Model transition-to-employment projects should establish employment support 
services. 

Indicators 

• Number of qualified job coaches or employment specialists available 

• Number of training hours provided to coworkers 

• Number of training hours provided to supervisors 

• Number of placements 

• Record of type of support provided at time of placement 

• Length of time on the job 

• List of services to be developed and expected function of services 

• Data from evaluation of roles or use of services 

• Data from needs assessment pertaining to support services 

Activity 10a Train job coaches. 

Indicators 

• Evaluation data from employers relevant to job coaches 

• Record of competencies identified for training (e.g., behavior management, 
environmental analysis, task analysis, designing accommodations, communication 
skills, etc.) 

• Number of hours of training provided 

• Documentation of materials used for instruction 

• Data from evaluation of instruction by job coaches 

• Evidence of competency checklist based on best practice criteria 

• Number of coaches trained; number employed 

• Performance data from observation of those trained pertaining to competencies 
(e.g., systematic fading, transferring support, trouble shooting) 

Activity 10b Provide coworker training. 

Indicators 

• Total number of coworkers trained 

• Number of coworkers trained as mentors 

• Number of coworkers trained as consultants 

• Number of coworkers trained as trainers or data collectors 

• Number of coworkers who advocate for the employee 

• List of questions coworkers have to use as prompts 

• Record of instructional support strategies offered or requested by coworkers 

• Data on coworker satisfaction with instructional support 

• Prc-post test of coworkers on instructional support utilization 

• Fidelity checklist based on student needs 

• Record of coworker use of systematic training strategies 

• Record of type and frequency of coworker support provided to employee 
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Activity 10c Utilize vocational rehabilitation counselors as appropriate. 

Indicators 

• Record of vocational rehabilitation participation on individual planning teams 

• Record of vocational rehabilitation participation on advisory board 

• Number of referrals to vocational rehabilitation 

• Number of vocational rehabilitation cases opened 

• Number of successful closures 

• Number of students receiving training provided by vocational rehabilitation 

• Record of services provided to students 

• Estimated cost of involvement per student 

• Frequency of contact by student, by school 

• Record of activities completed by counselors 

• Record of time extensions on funds for support 

• Record of funding by student or services provided 



Activity lOd Educate employers regarding benefits of hiring people with disabilities. 
Indicators 

• Number of training hours to employers on benefits of hiring people with 
disabilities 

• Number of presentations to business organizations, chambers of commerce, and 
service clubs regarding the benefits of employing persons with disabilities 

• Number of employers who have hired a person with a disability 

• Record of potential strategies for educating employers 

• Record of strategies chosen by employers 

• Number of presentations made to employer groups 

• Data pertaining to present status of employees with disabilities per employment 
site or sector 

• Record of and data from needs assessment conducted with employers 
Activity lOe Train and utilize job developers. 

Indicators 

• Number of job developers trained 

• Number of jobs developed 

• Record of competencies identified for training (e.g., conducting community job 
market surveys, contacting prospective employers, conducting analyses of job and 
work environments, developing client profiles, and job matching) 

• Number of jobs and type of information listed in job site file 

• Record of recruitment strategies used and list of those that are most successful 

• Record of job tasks and applicable instructional support strategies 

• Pre-post measure of task or job performance 

• Documentation of materials used for instruction 

• Evaluation data pertaining to instructional methods 

• Performance data of persons trained pertaining to competencies 

• Employer evaluation data pertaining to performance of job developers 
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PROGRAM LEVEL 

Outcome 1 1 Model transition-to-employment projects should undertake to develop materials to 
facilitate replication (e.g., replication guides, training manuals, assessment 
instruments). 

Indicators 

• Record of replication materials 

• Documentation of project activities and systems (to be incorporated into 
replication documents) 

• Record of comments from an "editorial" board or review committee 

• Record of people or projects to whom materials were sent 

• Information or descriptions from replication sites 

• Number of materials requested and distributed 

• Evaluation data of materials by users 

• Documentation of marketing process or activities 

• Documentation of demand (requests, orders) for materials 

• Documentation of known formal and informal replication attempts using the 
materials 

• Number and type of products developed 



Activity 11a Allocate a section of the project budget to publication and production costs. 

Indicators 

• Amount spent on publication or production 

• Itemized listing of associated costs 

• Percent funds allocated to publication and production 

• Percent of funds used 

• Record of budgeted allocations for publication 

• Documentation that budget was spent as specified 

Activity lib identify production priorities initially and monitor throughout the life of the project. 

Indicators 

• Data from process evaluation reports of project implementation activities 

• Quarterly reports on progress towards production goals 

• Evidence that final product completed by anticipated production date 

• Type and number of products by year of project 

• Evidence that production is included in evaluation plan 

• Record of timeline illustrating production schedule 
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PROGRAM LEVEL 

Outcome 12 Model transition-to-employment projects should demonstrate cost effectiveness. 

Indicators 

• Record of all activities and costs associated with project 

• Dollar reduction associated with less reliance on social programs 

• Ratio of cost of support to wages earned 

• Marginal program costs over and above average per pupil expenditure 

• Record and accounting of in-kind contributions or services provided by other 
agencies 

• Monetary value associated with services provided by volunteers 



Activity 12a Record all real costs of project activities. 

Indicators 

• Dollars allocated and spent on student training and support activities 

• Documentation of accounting system and procedures 

• Data or findings from audit reports 



Activity 12b Record effectiveness measures such as time allocated to training and quality of life 

measures. 

Indicators 

• Cost/benefit analysis data 

• Student earnings while participating in program 

• Reduction in costs of social programs (e,g., SSI, welfare, etc.) 
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ORGANIZATIONAL LEVEL 

Outcome 13 Model transition projecfc/should disseminate information about their projects by 

producing a product at least at the level of an article for the popular press. 

Indicators 

• Bibliography of articles or products (e.g., newsletters, monographs, replication 
materials, videotapes, brochures) 

• Number of or list of people or entities receiving products 

• Record of articles produced 

• Record of publication source and audience 

• List of journals, magazines, newspapers, TV shows in which program has been 
reported 

• Number of products by type 

• Number of presentations made pertaining to project 

• Evaluation information pertaining to products 

Activity 13a Employ a project director with a commitment to dissemination of information at 

least at this level. 

Indicators 

• Bibliography of articles or products 

• Evidence that vita of director reflects writing background 

• Evidence that job announcement reflects dissemination as a job responsibility of 
director 

• Evidence that job description reflects dissemination as a job responsibility of 
director 

• Articles written 

• Publication source and audience 

• Identification or evidence of key staff person or department responsible for 
dissemination activities 

• Evidence that director has documented performance in scholarship 

• Publication record of director 

• Number of project objectives pertaining to dissemination 

Activity 13b Require all workers to keep accurate records of all service and other activities 

conducted by the project. 

Indicators 

• Number of activities conducted by category and personnel 

• Evidence of recording forms 

• Program evaluation results pertaining to record keeping 

• Record of weekly activities log compiled by project staff 

• Bi-annual summary of activities per project 

• Project documentation notebook by category and by event 

• Record of products disseminated 

• Travel records of employees (e.g., miles, date, place) 

• Data from cross-checking records (e.g., service records per student with travel log 
of employee) 
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ORGANIZATIONAL LEVEL 

Outcome 14 Transition-to-eraployraent projects should develop and document a formal interface 
between education and community services (e.g., between schools and state 
vocational rehabilitation agencies). 

Indicators 

• Record of meeting held, at what level (state, regional, local), and agenda 

• Letters of formal agreement 

• Pre and post assessment of consumer and agency satisfaction 

• Record of formal interagency agreements 

• Research data on participation of community service providers in transition 
planning and implementation processes 

• Record of interagency inservice training and forums related to transition 

• Record of active transition councils with representatives from schools, community 
services, families, students, etc. 

Activity 14a Conduct workshops to train personnel. 

Indicators 

• Record of needs assessment to facilitate the interface 

• Record of number of needs or resources that can and cannot be met and 
justification 

• Record of workshops and participants involved to determine training content 

• Evidence of training packets including objectives and materials 

• Demographics of persons involved in workshops and agencies represented 

• Data from workshop evaluations 

• Research data on changes in transition activities following workshops 

• Number of workshops 

• Number of participants 

Activity 14b Communicate needs of project consumers to community agency personnel. 

Indicators 

• Documentation of attendance by community agency personnel at transition 
planning meetings 

• Research data on the outcomes of service provision related to the communication 
of needs 

• Documentation of planning for future services based on currently communicated 
needs 

• Data from consumer surveys 

• Data from needs assessment studies 
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Activity 14c Document meetings between education professionals and 

professionals/paraprofessionals outside of education. 

Indicators 

• Record of meeting dates, topics, attendance, and representation of groups 

• Research data related to outcomes of such meetings (e.g., impact on services 
available) 

• Record of meeting agendas 
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ORGANIZATION LEVEL 

Outcome 15 Transition-to-employment projects should develop and document a cooperative 

service delivery model where more than one agency is providing consumer services. 

Indicators 

• Number of agencies providing services 

• Organizational structure for service provision 

• Interagency agreement(s) 

• Record of referral among agencies 



Activity 15a Articulate the roles of all associated agencies. 

Indicators 

• Evidence of a process for the development of collaborative agreements 

• Documentation of collaborative agreements 

• Documentation of contractual arrangements 

• Evidence of a process to review roles on a regular basis 

• Number of formal operating agreements 

• Number of informal operating agreements 

• Evidence of networking effectiveness 

• Evidence of a designated "coordinating" agency to oversee local agencies 

• Number or existence of state laws or regulations reducing barriers to agency 
collaboration or articulating collaborative roles 

• Documentation of services provided by and efforts of each agency 



Activity 15b Employ personnel whose role is to coordinate project activities. 

Indicators 

• Job description of project coordinator or manager 

• Evidence of a "coordinating" agency to oversee local agencies 

• Documentation of state and local funds earmarked to support coordinating agency 
in this task 

• Evidence of agency and project funding of personnel 



Activity 15c Document services provided by cooperating agencies. 

Indicators 

• Record of employment services provided by type and by student 

• Record of community living facilities and/or services provided 

• Record of transportation arrangements and services 

• Assignment of coordinating personnel, agency, or local planning councils to collect 
specific data on services offered, clients, costs, etc. 

• Analysis of data collected for future decision making 

• Case history of clients 

• Number and type of activities completed by caseworkers 

• Evidence of a process for evaluating accountability of cooperating agencies 
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COMMUNITY LEVEL 

Outcome 16 Model transition-to-cmpioyment projects should demonstrate improved access to 
community-based services for persons with disabilities. 

Indicators 

• Number and type of generic resources used 

• Number and type of applications fded with community-based agencies 

• Number and type of community leisure activities participated in 

• Record of community resource training (banking, shopping for clothes or 
groceries, riding public bus, use of restaurant or fast-foods, etc.) 

• Record of activities (contacts made) to improve access 

• Evidence of agreements developed 

• Pre and post survey data pertaining to community access 

• Record of projects or products completed 

• Record of pre- and post-project employment, recreation, and living sites 

• Number of community-based activities 

• Number of hours in community-based activities 

• Data pertaining to pre- and post-satisfaction of persons with disabilities and their 
families 

• Data pertaining to increase in options or services available in the community (e.g., 
wheel chair accessible shopping carts, visual aids, ramps, etc.) 

• Data pertaining to increased public awareness of people with disabilities 

Activity 16a Document the number of students in the project served by community agencies. 

Indicators 

• Record of agencies contacted and result of contact 

• Record of identified needs not served by community agencies 

• Number referrals made and percent accepted 

• Number of students served by community agencies pre and post project and ratio 
of students with and without disabilities to staff 

• Record of products developed identifying agencies and services 

Activity 16b Conduct outreach activities such as seminars and workshops for community agency 

personnel. 

Indicators 

• Number and type of seminars 

• Number of people attending seminars 

• Evaluation data pertaining to the seminars 

• Number and type of contacts to community agencies (mailings, phone calls, etc.) 

• Number of requests for information 

• Number and type of agencies contacted 

• Record of workshops, agendas, and agency personnel who attended 

• Number of workshops completed 

• Number of participants 

• Data pertaining to satisfaction of participants 

• Evidence of workshop materials (e.g., program, registration materials) 

5? 
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Activity 16c Conduct technical assistance services to center-based agency personnel to foster 

conversion to community-based services. 

Indicators 

• Number and type of information disseminated 

• Amount of time per staff person spent in consultation activities 

• Number and types of requests for technical assistance 

• Record of consultation activities and associated outcomes 

• Time spent per agency 

• Pre and post technical assistance competencies and performance evaluations of 
agency personnel 

• Data pertaining to agency needs necessary for the conversion of services 

• Evidence of technical assistance materials (e.g., recommended reading or video 
list, manuals) 

• Pre and post measures of the number of agency clients participating in center- 
based and community-based services 
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COMMUNITY LEVEL 

Outcome 17 Model transition-to-eraployraent projects should demonstrate improved work 
opportunities for youths with disabilities. 

Indicators 

• Record of types of jobs which comprise placements 

• Statistics pertaining to wages, benefits, and hours 

• Percent of graduating class employed by level of employment (i.e., full time, part 
time) 

• Percent employed at or above minimum wage 

• Percent who move to improved work situations (e.g., promotions, job changes for 
increased salaries, benefits, working hours, etc.) 

• Percent who lose jobs and/or move to "poorer" jobs 

• Number of employers associated with project or program 

• Number of students employed first year of project compared to subsequent years 

Activity 17a Evaluate and document effectiveness of job placement and maintenance activities. 

Indicators 

• Length of time on job 

• Employee satisfaction data concerning job placement, pre and post project 

• Employer satisfaction data concerning job placement, pre and post project 

• Documented opportunities for advancement 

• 1, 3, 5 year follow-ups on youths: 

1. Employed in jobs for which training was provided; in jobs for which training 
was not provided; 

2. Employed but changed job (up and down) in job trained; not in jobs trained 

3. Unemployed; never employed; previously employed 

• Data on youths employed and wages, benefits, length of employment, pre and post 
project 

• Data on family satisfaction with job placement, pre-post project 

• Data on employer willingness to hire, pre-post project 

Activity 17b Research job trends and business requirements. 

Indicators 

• Record of project or employer advisory committee, members, meetings 

• Labor market surveys: 

Stable employment opportunities 

Potential increased employment opportunities 

Decreasing employment opportunities 

Dead-end employment 

Career ladder employment 

Job requirements 
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Activity 17c Work cooperatively with community agencies to conduct longitudinal studies* 
Indicators 

• Assess student outcomes: employment, community living, recreation, etc. 

• Assess quality of life via self rating or reliable informant 

• Follow-up data on individuals who change agencies 

• Evaluation data pertaining to working relationship between agencies and project 

• Record of agreements detailing longitudinal studies to be conducted and roles of 
participants 

• Documentation of funding source 

• Evidence of research design for study 



From: Kohler, P.D., & Rusch, F.R. (1993). School to work transition: Identification of employment- 
related outcome and activity indicators. Champaign: University of Illinois, Transition Research 
Institute. (Submitted for publication.) 
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